Skip to content

fix: keep bootstrapRef across the VM creation request-id patch - #800

Merged
thunderboltsid merged 2 commits into
mainfrom
issue/fix-bootstrapref-nil-panic
Oct 6, 2026
Merged

thunderboltsid merged 2 commits into
mainfrom
issue/fix-bootstrapref-nil-panic

Conversation

@vijayaraghavanr31

@vijayaraghavanr31 vijayaraghavanr31 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • The VM creation request-id patch was applied to rctx.NutanixMachine itself, so the client overwrote it with the API server's response and dropped every in-memory change not yet persisted: spec.bootstrapRef, the finalizer added at the top of reconcileNormal, and the deprecated finalizer removal.
  • addGuestCustomizationToVM then panicked on bootstrapRef.Kind during the first VM create. The retry succeeded (the request id already existed, so no patch), which is why scale tests logged the panic and the cluster still came up.
  • The request-id patch is now applied to a copy; the live object keeps its unpersisted changes and the deferred patch at the end of the reconcile persists them as before. Both guest-customization helpers return an error when the ref is nil instead of panicking.

How has this been tested?

  • Unit: go test ./controllers/ (new fake-client test asserts bootstrapRef and the finalizer survive the mint patch; nil-ref guards tested). Both new tests fail without the controller change.
  • envtest: confirmed against a real API server that the pre-fix patch drops the in-memory bootstrapRef, finalizer, labels and status, and that with the fix the deferred patch persists the finalizer, bootstrapRef and request id.
  • e2e: ran the quickstart spec (quickstart && !clusterclass, cilium) against a dev PC on both upstream/main and this branch and inspected the dumped CAPX manager logs:
    • main: 2 Observed a panic in addGuestCustomizationToVM, one per machine on the first create; VMs created on the retry.
    • this branch: 0 panics; both VMs created in the same reconcile as the request-id mint. The NutanixMachine objects have the request-id annotation, finalizer and spec.bootstrapRef persisted, and nodes joined.

@codecov

codecov Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 54.54%. Comparing base (883f3c9) to head (7dc8a21).
⚠️ Report is 131 commits behind head on main.

Files with missing lines Patch % Lines
controllers/nutanixmachine_controller.go 91.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #800       +/-   ##
===========================================
+ Coverage   43.96%   54.54%   +10.57%     
===========================================
  Files          18       32       +14     
  Lines        2593     6296     +3703     
===========================================
+ Hits         1140     3434     +2294     
- Misses       1417     2546     +1129     
- Partials       36      316      +280     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The request-id patch was applied to rctx.NutanixMachine itself, so the
client overwrote it with the API server's response and dropped every
in-memory change not yet persisted: spec.bootstrapRef, the finalizer
added at the top of reconcileNormal, and the deprecated finalizer
removal. Patch a DeepCopy instead so those stay on the live object and
are persisted by the deferred patch at the end of the reconcile, which
makes the bootstrapRef special case in the baseline unnecessary.
@thunderboltsid
thunderboltsid merged commit db211ad into main Oct 6, 2026
29 checks passed
@thunderboltsid
thunderboltsid deleted the issue/fix-bootstrapref-nil-panic branch October 6, 2026 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants